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Abstract — Raptor codes are rateless codes that achieve the 
capacity on the binary erasure channels. However the maximum 
degree of optimal output degree distribution is unbounded. 
This leads to a computational complexity problem both at 
encoders and decoders. Aref and Urbanke investigated the 
potential advantage of universal achieving-capacity property of 
proposed spatially-coupled (SC) low-density generator matrix 
(LDGM) codes. However the decoding error probability of SC- 
LDGM codes is bounded away from 0. In this paper, we 
investigate SC-LDGM codes concatenated with SC low-density 
parity-check codes. The proposed codes can be regarded as SC 
Hsu-Anastasopoulos rateless codes. We derive a lower bound of 
the asymptotic overhead from stability analysis for successful 
decoding by density evolution. The numerical calculation reveals 
that the lower bound is tight. We observe that with a sufficiently 
large number of information bits, the asymptotic overhead and 
the decoding error rate approach with bounded maximum 
degree. 

I. Introduction 

Spatially-coupled (SC) low-density parity-check (LDPC) 
codes attract much attention due to their capacity-achieving 
performance under low-latency memory-efficient sliding- 
window belief propagation (BP) decoding. The studies on 
SC-LDPC codes date back to the invention of convolutional 
LDPC codes by Felstrom and Zigangirov [1]. Lentmaier et 
al. ! observed that the BP threshold of regular SC-LDPC codes 
coincides with the maximum a posterior (MAP) threshold of 
the underlying block LDPC codes with a lot of accuracy by 
density evolution [2]. Kudekar et al. proved that SC-LDPC 
codes achieve the MAP threshold of BEC [3] and the binary- 
input memoryless output-symmetric (BMS) channels [4] under 
BP decoding. 

Rateless codes are a class of erasure-recovering codes which 
produce limitless sequence of encoded bits from k information 
bits so that receivers can recover the k information bits from 
arbitrary (l + a)fc/(l — e) received symbols from BEC(e). We 
denote overhead by a. Designing rateless codes with vanishing 
overhead is desirable, which implies the codes achieve the 
capacity of BEC(e). LT codes [5] and raptor codes [6] are 
rateless codes that achieve vanishing overhead a — !■ in 
the limit of large information size over the BEC. By a nice 
analogy between the BEC and the packet erasure channel (e.g., 
Internet), rateless codes have been successfully adopted by 
several industry standards. 

A raptor code can be viewed as concatenation of an outer 
high-rate LDPC code and infinitely many single parity-check 
codes of length d, where d is chosen randomly with probability 
Vld for d>l. Raptor codes need to have unbounded maximum 



degree d for Vtd 7^ 0. This leads to a computation complexity 
problem both at encoders and decoders. 

The authors presented empirical results in [7] showing that 
SC MacKay-Neal (MN) codes and SC Hsu-Anastasopoulos 
(HA) codes achieve the capacity of BEC with bounded maxi- 
mum degree. Recently a proof for SC-MN codes are given in 
[8]. It was observed that the SC-MN codes and SC-HA codes 
have the BP threshold close to the Shannon limit in [9] over 
BMS channels. 

Aref and Urbanke [10] investigated the potential advantage 
of universal achieving-capacity property of SC low-density 
generator matrix (LDGM) codes. They observed that the 
decoding error probability steeply decreases with overhead 
a = with bounded maximum degree over various BMS 
channels. However the decoding error probability was proved 
to be bounded away from with bounded maximum degree 
for any a. This is explained from the fact that there are a 
constant fraction of bit nodes of degree 0. 

In this paper, we investigate SC-LDGM codes concatenated 
with SC-LDPC codes. The proposed codes can be regarded 
as SC-HA rateless codes. We derive a lower bound of the 
asymptotic overhead from stability analysis for successful 
decoding by density evolution. The numerical calculation 
reveals that the lower bound is tight. We observe that with 
a sufficiently large number of information bits, the asymptotic 
overhead and the decoding error rate approach with bounded 
maximum degree. 



II. Encoder and Decoder 



A. Encoder 



Let k denote the number of information bits. We define a 
{di,dr,dg, L,w) code for di >2,dr >2,dg > 2 as follows. 
The {di, dr,dg, L, w) code are defined on L sections from to 
L — 1. Each section has M pre-coded bits. Note that, in [3], 
2L + 1 sections [—L, +L] were considered. Instead, for the 
sake of simplicity, we consider L sections in [0, L — 1]. First, 
the k information bits are pre-coded with {di, dr, L, w) codes 
[3] into LM bits a;(0, 0), . . . , x{L - 1, M - 1). In this paper, 
we assume that the bits in the i-th section for i G [0,i — 1] 
are transmitted and the bits in other sections are shortened. 
Namely, the shortened bits are set to and are not transmitted. 
Let Rprc{L) denote the design coding rate of {di,dr,w,L) 
codes. In [3], Rpic{L) is given by 
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It follows that k Rp^c{L)LM. 

After encoding the k bits into LM coded bits by pre-code, 
the LM pre-coded bits further will be encoded by an inner 
code as follows. Repeat the following procedure endlessly for 

t e [1, oo). 

1) Choose a section i*^*) G [0,L + w — 2] uniformly at 
random from L + w — 1 sections. 

2) Choose dg section shifts . . . , j^*"* e [0,w — 1] with 
repetition uniformly at random. 

3) Choose dg bit-indices \ . . . , e [0,M - 1] with 
repetition uniformly at random. 

4) Add dg bits and transmit the sum as 



B. Decoder 

Assume that transmission takes place over BEC(e) and we 
have n received symbols y'^', . . . , y'"^ each of which is 0, 1 
or '?'. Define the overhead a as 
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In this setting, we have {l + a)k = 7i(l — e) unerased received 
symbols. Independence of the coding scheme ensures that we 
can assume, without loss of generality, that time indices of n 
received symbols are arbitrary. For simplicity, we assume that 
the receiver receives n symbols at time t = 1, . . . ,n without 
loss of generality. 

We assume that the decoder knows dg section shifts 

,(t) 



I), in ^ for each 



and bit-indices /J*'',., 
received symbol at time t = l,...,n. From these infor- 
mation and the knowledge of the precode, one can con- 
struct a factor graph for sum-product decoding [11]. The 
factor graph consists of LM variable nodes (bit nodes) 
a;(0, 0), . . . , x{L - 1, M - 1) and (1 - Rp,c{L))LM parity- 
check factor nodes (check nodes) of pre-code and factor nodes 
(channel nodes) of factor 

- jt\ jf ) + • ■ • + - ^t^ji?) = y'*^] (2) 

for t = 1 , . . . , n, where 1 [ • ] is defined as 1 if the argument 
is true and otherwise. We say that the factor node of factor 
(|2]l is in the section i^*). 

III. Performance Analysis 

In this section, we investigate the performance of the 
coupled rateless codes and derive a bound. 

A. Performance Analysis by Density Evolution 

In this subseciton, we derive the density evolution update 
equation. The following lemma clarifies the degree distribu- 
tions of inner codes. 

Lemma L Let be the probability that a bit node has d 
neighboring channel nodes. Let /3 be the average number of 



channel nodes adjacent to a bit node. In the limit of large M, 
we have 
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Proof. Let N denote the average number of channel nodes 
per section. There are L + w 1 sections containing channel 
nodes. We have n channel nodes in total. 



N = 
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where we used k ~ Rprc{L)LM. Recalling that (3 is the 
average number of channel nodes adjacent to a bit node, we 
have 

d_^ 

^ M 

Equation (O immediately follows from this. Each section has 
N channel nodes of degree dg, in other words, we have dgN 
edges in each section. Let denote the probability that 
a bit node in the i-th section has d channel nodes within 
sections from i to i + w — 1. Since each channel node is 
generated independently, the probability A^; follows a binomial 
distribution as follows. 
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The probability generating function of A^ is given as follows. 
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exp[-/3(l - x)] = 
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This implies A^^ = ^ ^| in the limit of M oo. In other 
words, the degree d follows the Poisson distribution of average 
/3. □ 

Let us describe density evolution update equations. Let p) 
and s] be the erasure probability of messages sent from 
bit nodes in the i-th section to check nodes and channel 
nodes, respectively, at the ^-th iteration of BP decoding of 
{di,dr,dg,L,w) codes in the limit of large AL The density 
evolution [12] gives update equations for p'^' and sf'' as 



follows. For i ^ [0,L — 1], p. 



= 0. Forie [0,i-l], 



1, and for £ > 0, 
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where 
Let 



3=0 
_ A'ix) 



k=0 



^(•^) = ^=exp[-/3(l-.T)]=A(.T). 



be the decoding error probabiUty at the ^-th 



iteration of BP decoding given as follows. 
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Definition 1: One can easily check 



has its limit 



])(oo)/- 



>ooPtf''(i) since PJ^'^'' is decreasing in 
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We define overhead threshold a\ and its corresponding /?£ as 
follows. 

al :=inf{a>0|P[,°°^(L)=0}, 
/32 :=inf{/3>0|P[,°°)(i)=0}. 

We say {di, dr, dg, L, w) codes achieve the capacity o/BEC(e) 
if 

limsupa^ ^ 0- 
Discussion 1: We will explain why we exclude the case 
dg = \. Assume dg = 1. The density evolution update 
equations can be reduced as follows. 
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(zG [0,L-1]), 
(z^ [0,L-1]). 



This is equivalent to the density evolution update equation 
of the precode that is a {di,dr,w, L) code transmitted over 
BEC(A(e)) [3]. If the error probability goes to 0, A(e) has to be 
less than the Shannon limit A(e) = e^^^-^^^^' < 1 - i?pi.o(-L). 
It follows that PI is bounded as follows. 
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From ^ we have 
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This implies the {di,dr,dg = l,L,w) codes do not achieve 
the capacity of BEC(e). This is the reason why we exclude 
the case = 1 in this paper 

Lemma 2: The {di,dr, dg, L, w) codes achieve the capacity 
of BEC(e) if and only if 

limsup/32.-i^(l-J). 
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Proof. This is straightforward from (|3), we have 
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B. Performance Bound by Stability Analysis 

In the following theorem, we derive a lower bound of 
overhead threshold a^. 

Theorem 1: For [di 
then there exist and 



2, dr, dg, L, w) codes, if 
such that 
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Proof: Let Pl denote an i x L matrix whose {i, j) entry is 

'■ ■ -) 

-. As we will see, this does not depend on £. Let p{Pl) 



dp'; 



denote the spectral radius of Pl ■ We will derive a lower bound 

of p{Pl). 

Some calculation reveals that at p'^' = s'^^ = for di = 2. 
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and 
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= for di > 2. It holds that for di > 2, 

dpf^'^ dsf^'^ dsf+'^ 



ds 



dp 



ds^^ 



= 0. 



at p 



s^^) = 0. We drop i since ^ is independent of £. 
From (|5]l, we can see that Pl is a positive band matrix of 
width w, which is defined in Definition |4] in Appendix. Since 
Pl is a positive band matrix of width w, one can see that Pl 
is irreducible from Lemma |4] in Appendix. Let Ai, . . . , Al be 
the eigenvalues of Pl, recall that p{Pl) is the spectral radius 
of Pl- We have 

p{Pl) ■■= max(|A,,|). 

i 

Since Pl is symmetric, the eigenvalues are real. 

Let Al > ... > Al be the eigenvalues of Pl- Perron- 
Frobenius theorem [13] asserts that the eigenvalue that gives 
the spectral radius of a non-negative irreducible matrix is pos- 
itive. Since Pl is non-negative symmetric irreducible matrix, 
the eigenvalue that gives spectral radius of Pl is positive. Then 
we have 



p{Pl) = Al. 



(6) 



For (5 > 0, we define /3 + 5. Since /3 > /3l, it follows 

^i°°\L) = 0. From ©, we have for Va; e \ {0}, 
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where we used [14, Theorem 4.2.2] for (a). Solving f3 from 
this inequality, we obtain 
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lim^^o P PI denote that /?£ > RHS of (HI). A trivial lower 
bound > is true, since we can not surpass the capacity. 
From this and (|4|l, it follows that 
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In the limit of large L, we have 
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This concludes Theorem [T] 

Discussion 2: For L > 2w — l, have entries taking value 
from 1 to w. From [14, Lemma 5.6.10], we can bound p{Pl) 
as follows. 



p(Pl)<||Pl||i:= max f]l(^Lk,l 

l<i<L ^ — ^ 

= (d„-l)e"''(-')-l(. + 2^ 



= (d, - l)e 



From this, we can see that the bound (|7]i is tight for large L. 

□ 

Corollary 1: For capacity-achieving [di = 2, dr, dg, L,w) 
codes have to satisfy 



dg > 



dr ln{dr — 1) 



(9) 



This condition is not satisfied for dr — 2 or dg — 2. 

Proof: From Definition [T] capacity-achieving codes satisfy a*j^ 

goes to in the limit of large L. To be precise. 



lim = max 
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- 1,0 



= 0. 



The inequality ^ immediately follows from this. 




Fig. 1 . The asymptotic overhead and the average degree of /3£ and their 
lower bounds and /J^ of {dl = 2,dr = 3, dg = 2, L,w = 2) codes over 
BEC(e=0.5). The asymptotic overhead tlireshold a^^ does not converge to 
since the codes do not satisfy the condition of Corollary [T] Figure suggests 
the lower bounds are tight for large L. 



IV. Decoding Performance 

In this section, we demonstrate the decoding performance 
of the {di,dr,dg,L,w) codes. 

Figure [T] shows convergence the overhead threshold and 
/?£ and their lower bounds aj^ and f3*^ of (di ^ 2,dr ^ 3,dg ^ 
2,L,w = 2) codes over BEC(e=0.5). The codes do not satisfy 
the condition of Corollary [T| This explains why does not 
converge to and /?£ does not converge to 4/3 which is given 
in Lemma 12] as the limiting value of capacity-achieving codes. 



_ 31n(2)-2 



2 



0.03972 
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We observe that approaches a* 
and 131 approaches = 21n(2) ~ 1.38629 which suggest 
the lower bounds are tight for large L. 

Figure |2] shows convergence the asymptotic overhead thresh- 
old a*j^ and the average degree of (31 and their lower bounds 
and /3*^ of {di = 2,dr = 3, dg = 3, L,w = 2) codes over 
BEC(e=0.5). The codes satisfy the condition of Corollary [T] 
Thought this does not necessarily ensure approaches and 
/3£ approaches 2 which is given in Lemma |2] as the limiting 
value of capacity-achieving codes, this is likely the case. We 
observe that approaches a^^ = and /32 approaches 
f3* = 2, which suggest the lower bounds are tight for large 

Figure [3] compares approaching speed of overhead threshold 
of {di = 2,dr,dg = 3,L,w = 2) codes with dr € 
{3, 4, 14, 15, 20, 30} over BEC(e=0.5). The codes of dr < 14 
satisfy the condition of Corollary[T] while the codes of dr > 14 
do not. The fastest approaching speed is attained at dr — 14. 

V. Conclusion 

We propose spatially-coupled precoded regular rateless 
codes. We have derived a lower bound a*j^ of asymptotic 
overheads threshold a^. The numerical calculation of density 
evolution shows that the bound is tight for large coupling 
number L and asymptotic overheads threshold a*j^ goes to 
for large L with bounded density. The possible future work 




Fig. 2. The asymptotic overhead and the average degree of /3£ and 



and /3* of (d; 



2,dr- 



3,( 



3,L, 



2) 



codes over BEC(e=0.5). The codes satisfy the condition of Corollary [T] We 
observe that approaches a*^ = and {3*^ approaches /?* = 2, which 
suggest the lower bounds are tight for large L. 




Fig. 3. Comparison of approaching speed of overhead threshold a*^ of 
(d, = 2,dr,dg = 3,L,w = 2) codes with dr G {3,4,14,15,20,30} 
over BEC(e=0.5). The codes with dr < 14 satisfy the condition of Corollary 
[T] while the codes with d^ > 14 do not. The fastest approaching speed is 
attained at d,, = 14. 



is an extension to BMS channels and a proof for capacity- 
achievability. 
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Appendix 

Definition 2: A square matrix A is said to be a reducible 
matrix when there exists a permutation matrix P such that 

fx Y\ 

P^AP = ( Q ^ j , where X and Z are both square. 

Otherwise A is said to be irreducible. 

Definition 3: Let A be a square matrix of size m. The graph 
G{A) of A is defined to be the directed graph on m nodes 
A^i, Nm in which there is a directed edge leading from Ni 
to Nj if and only if ^ 0. Qi^A) is called strongly connected 
if for each pair of nodes {Ni,Nj) there is a sequence of 
directed edges leading from Ni to Nj. 
The following lemma can be found in [14, p. 362]. 

Lemma 3: A square matrix A is an irreducible matrix if 
and only if Q{A) is strongly connected. 

Definition 4: We say that a square real matrix A = (ai.j) 
is positive band matrix of width w if 

1=0 {\j\>w) 
Lemma 4: L x L matnx Al is irreducible if Al is positive 
band matrix of width w > 1. 

Proof: From Definition |4] it holds that for any < .j < L, 
Nj e Q{A) is adjacent to N, 



The lemma follows readily from Lemma |3] 



max(0,j-l) ™d iVjjjij,(j^i^i_i). 



